The present invention relates to methods of imaging and analysis of particles and, in particular, to a method for in-situ focus-fusion multi-layer spectral imaging and analysis of particulate samples.
When a sample featuring, for example, a particle, an aggregate of particles, or a dispersion of particles, has large layer or depth variations relative to changes in the distance from which it is viewed, an image of the sample exhibits a layer dependent or spatially varying degree of sharpness. This is referred to as a defocused image of the sample or scene, where some of the objects of the scene are in focus, while other objects of the scene are out of focus. Defocused images contain information potentially useful for scene analysis. The analysis of scenes from defocused images is of general interest in machine vision applications, for example, in active vision or robot vision where a camera actively explores a scene by continuously changing its position or field of view, relative to scene features. Applying scene analysis to defocused images is highly useful for accurately interpreting and understanding images of pharmaceutical, biomedical, biological, environmental, and microscopy samples, where layer or depth variations of imaged samples of powders, frozen suspensions of powders, biological specimens, air pollution particulates, or other multi-layered particulate samples are typically large compared to imaging distances. Scene analysis of defocused images is of particular applicability to depth dependent particulate samples, where, for instance, one or more layers of bacterial or fungal growth, exhibiting fluorescent emission properties in addition to the fluorescent emission properties of the particles themselves, is present on the particles, and there is a need for separation of imaging and analysis of the bacterial or fungal growth from that of the particles. Additionally, scene analysis is particularly applicable to depth dependent particulate samples of aerosols containing polycyclic aromatic hydrocarbons (PAHs) and other fluorescent particulate contaminants.
In conventional scene analysis using methods and systems for imaging particles, for example, for each scene, there is auto-focusing, where a best focal position is determined for use in analyzing or classifying particle properties. For some scenes, this is possible, and a focused image may be obtained in an automatic manner. Typically, an auto-focus module is coupled with a computer controlled mechanism that automatically changes the focal position, by moving along an axis parallel to the optical axis of the imaging or focusing sensor, thereby enabling identification of a good focal position. For other scenes, a good focal position is not guaranteed to exist and further image processing based on focus-fusion methodology is required.
When a focused image of a spatially varying or depth dependent scene can not be generated by using such electromechanical microscopy means, such that a single focal position can not be identified, a focused representation of the scene can be constructed by combining or fusing several defocused images of the same scene. This process is referred to as focus-fusion imaging, and the resulting image of such processing is referred to as a focus-fusion image. Defocused images, for example, those acquired during auto-focusing, are fused together such that each target in a given scene is in correct focus. Scene targets are detected by analyzing either the focused image, if it exists, or the focus-fusion image.
A current technique of imaging particles is based on spectral imaging. In spectral imaging, a particulate sample is affected in a way, for example, excitation by incident ultraviolet light upon the sample, which causes the sample to emit light featuring an emission spectra. Emitted light is recorded by an instrument such as a scanning interferometer that generates a set of interferogram images, which in turn are used to produce a spectral image, also referred to as a cube image, of the sample. Each cube (spectral) image is a three dimensional data set of voxels (volume of pixels) in which two dimensions are spatial coordinates or position, (x, y), in the sample and the third dimension is the wavelength, (xcex), of the imaged (emitted) light of the sample, such that coordinates of each voxel in a spectral image or cube image may be represented as (x, y, xcex). Any particular wavelength, (xcex), of imaged light of the sample is associated with a set of cube images or spectral fingerprints of the sample in two dimensions, for example, along the x and y directions, whereby voxels having that value of wavelength constitute the pixels of a monochromatic image of the sample at that wavelength. Each cube image, featuring a range of wavelengths of imaged light of the sample is analyzed to produce a two dimensional map of the chemical composition, or of some other physicochemical property of the sample, for example, particle size distribution.
An example of a method and system for real-time, on-line chemical analysis of particulate samples, for example, polycyclic aromatic hydrocarbon (PAH) particles in aerosols, in which the PAH sample is excited to emit light, for example fluorescence, is that of U.S. Pat. No. 5,880,830, issued to Schechter, and manufactured by GreenVision Systems Ltd. of Tel Aviv, Israel, and is incorporated by reference for all purposes as if fully set forth herein. In the disclosed method, spectral imaging techniques are implemented to acquire an image and analyze the properties of fixed position PAH particles. As part of this method, air is sampled by means of a high volume pump sucking a large volume of air featuring aerosol contaminated with PAH particles onto a substrate, followed by on-line imaging and scene analysis of the stationary particles.
A method of calibration and real-time analysis of particles is described in U.S. Pat. No. 6,091,843, to Moshe et al., and is incorporated by reference for all purposes as if fully set forth herein. The method described, is based on using essentially the same system of U.S. Pat. No. 5,880,830, for acquiring spectral images of static particles on a filter. Targets are identified in static particle images and are classified according to morphology type and spectrum type. Each target is assigned a value of an extensive property. A descriptor vector is formed, where each element of the descriptor vector is the sum of the extensive property values for one target class. The descriptor vector is transformed, for example, to a vector of mass concentrations of chemical species of interest, or of number concentrations of biological species of interest, using a relationship determined in the calibration procedure. In the calibration procedure, spectral images of calibration samples of static particles having known composition are acquired, and empirical morphology types and spectrum types are inferred from the spectral images. Targets are identified in the calibration spectral images, classified according to morphology type and spectrum type, and assigned values of an extensive property. For each calibration sample, a calibration descriptor vector and a calibration concentration vector is formed. A collective relationship between the calibration descriptor vectors and the calibration concentration vectors is found using chemometric methods.
In the method of U.S. Pat. No. 6,091,843, standard spectra are determined empirically in the calibration procedure. In such analytical procedures, empirical calibration is quite important for leading to highly accurate results based on image analysis and classification, because spectra of adsorbed chemical species in general, and, of PAHs in particular, are known to be altered by the surfaces on which they are adsorbed, and by the presence of contaminants during sample preparation and image acquisition. Moreover, in the described method, the relationship between the descriptor vector and the concentration vector accounts explicitly and simultaneously for both morphologies and empirically determined spectra. This is particularly important in cases where fluorescence spectra of crystal particles are known to depend on crystal morphology, in general, and crystal size, in particular.
Spectral imaging of spatially varying, depth dependent, or multi-layered samples of particles is not described in the above referenced methods and systems. Imaging and image analysis of a random single two-dimensional layer of a sample including particles are ordinarily straightforward. However, multi-layer imaging and image analysis of depth dependent particulate samples, for example, multi-layered dry particles, or particles in a frozen or immobilized suspension, are substantially more complex, for the reasons stated above. More often than not, images obtained of such particulate samples are defocused, and require special image processing techniques, such as focus-fusion, for obtaining useful information about the samples. Nevertheless, there are instances where it is necessary to obtain property and classification information of depth dependent particulate samples, in-situ, for example, as part of sampling an industrial process. In principle, a sample of dispersed or multi-layered particles is amenable to three-dimensional imaging and scene analysis. In practice, however, for depth dependent samples of particles, spectral imaging as presently practiced would involve tedious methodologies and system manipulations, making acquisition of high resolution images impossible or at best impracticable.
Scene analysis by applying focus-fusion methodology to defocused images acquired by multi-layer spectral imaging of depth dependent particulate samples would be quite useful for detecting and classifying in-situ physicochemical information of the particles, such as particle size distribution, morphological features, including structure, form, and shape characteristics, and chemical composition, which ideally involve multi-layer three-dimensional image analysis. For fusing defocused images, current focus-fusion procedures and algorithms typically involve information and parameters relating only to the extent to which acquired images are either focused or defocused, without inclusion of additional information and parameters specifically relating to particular properties and characteristics of the imaged object or sample, and relating to the information and parameters of the spectral imaging process. Characteristic sample physicochemical and spectral information and parameters can be quite relevant to imaging particulate samples, and ought to be included in a method of focus-fusion of acquired images of such samples. This is especially the case for images of particulate samples featuring layer dependent or spatially varying degree of sharpness. There is thus a recognized need for, and it would be highly advantageous to have, a method for in-situ focus-fusion multi-layer spectral imaging and analysis of depth dependent particulate samples.
The present invention relates to a method for in-situ focus-fusion multi-layer spectral imaging and analysis of depth dependent particulate samples. A unique method of focus-fusion is applied to focused and defocused images acquired from multi-layer spectral imaging of a depth dependent particulate sample, in order to construct focused fused cube (spectral) image representations of the imaged particles, thereby generating a focused image of essentially each particle in the sample. The method of the present invention introduces the use of a uniquely defined and calculated focus-fusion factor parameter, Fb, which combines (1) empirically determined particle physicochemical information and parameters relating to (i) particle chemical composition and associated chemistry, and relating to (ii) particle morphology such as particle size and shape, with (2) empirically determined particle spectral information and parameters such as (i) pixel intensity, (ii) signal-to-noise ratio (S/N), (iii) image sharpness, (iv) spectral distances, and (v) spectral fingerprints relating to spectral emission patterns of individual particles. The focus-fusion factor parameter, Fb, is used in critical steps of image detection, image analysis, and in algorithms for classification of particle characteristics. This uniquely determined parameter enables achievement of high levels of accuracy and precision in detection and classification of the sample, in general, and of the featured particles, in particular.
The method of the present invention includes collecting and analyzing physicochemical and multi-layer spectral data relating to the particles in the sample, including mapping of three-dimensional positions of particles, particle sizes, and characteristics of particle emission spectra. Scene information, in the form of spectral fingerprints, used in the analysis of focus-fusion of the multi-layer spectral images is further processed in order to generate relevant in-situ physicochemical information of the particles, such as particle size distribution, morphological features, including structure, form, and shape characteristics, and chemical composition. The focus-fusion multi-layer spectral image analysis includes a sophisticated classification procedure for extracting, on-line, useful information relating to particle properties and characteristics needed for generating a report applicable to monitoring or controlling an industrial process.
The method of the present invention enables multi-layer spectral imaging, multi-layer scene analysis, and multi-layer physicochemical characterization of particulate samples featuring depth dependency, which until now has not been described in the prior art of spectral imaging technology or of focus-fusion technology. Implementing the present invention is highly useful for accurately interpreting and understanding images of pharmaceutical, biomedical, biological, environmental, and microscopy samples, where layer or depth variations of imaged samples of powders, frozen suspensions of powders, biological specimens, air pollution particulates, or other multi-layered particulate samples are typically large compared to differential imaging distances.
In the pharmaceutical industry, applying the method for in-situ multi-layer focus-fusion spectral imaging and analysis of particulate samples is of particular applicability to depth dependent particulate samples of powders, where, for instance, one or more layers of bacterial or fungal growth, exhibiting fluorescent emission properties in addition to the fluorescent emission properties of the particles themselves, is present on the particles, and there is a need for separation of imaging and analysis of the bacterial or fungal growth from that of the particles. Additionally, the present invention is very well suited for analyzing defocused images of multi-component particulate samples of medicines, for example, medicines containing both active and inactive ingredients, whereby there is distinguishing and characterizing physicochemical properties and features of the active and inactive ingredients. In the environmental field of analyzing, monitoring, and controlling air pollution, the present method for focus-fusion spectral imaging scene analysis is particularly applicable to depth dependent particulate samples, such as airborne aerosols containing polycyclic aromatic hydrocarbons (PAHs) and other fluorescent particulate contaminants.
According to the present invention, there is provided a method for multi-layer imaging and analyzing a sample featuring particles, imaged particles exhibiting a spatially varying degree of sharpness, the method comprising the steps of: (a) providing a spectroscopic imaging system, including a sample holder movable by a three dimensional translation stage; (b) selecting and defining imaging scenario parameters for acquiring and analyzing spectral images of the sample, the imaging scenario parameters are particle physicochemical characteristics relating to particle chemical composition and particle morphology, and, particle spectral characteristics relating to pixel intensity, signal-to-noise, image sharpness, spectral distances, and spectral fingerprints relating to spectral emission patterns of individual particles; (c) adjusting and setting the spectroscopic imaging system for imaging at a selected field of view, FOVi, having central (x, y) position coordinates; (d) acquiring a cube (spectral) plane image of the sample in the selected field of view at a selected differential imaging/focusing distance, xcex94zij, by focusing the imaging system in z-direction until receiving a sharp gray level image of the sample; (e) constructing and analyzing a focused cube (spectral) plane image of the sample for an i-th field of view, FOVi, at a j-th differential imaging/focusing distance, xcex94zij, from the cube (spectral) plane image of the sample acquired in the step (d), whereby the constructing and analyzing are based on using a combination of the selected and the defined particle physicochemical and particle spectral imaging scenario parameters of the step (b); (f) repeating the step (d) and the step (e) in the same field of view, FOVi, for a selected range of imaging distance defined along the z-direction between the sample light illumination source of the imaging system, acquiring a plurality of the cube (spectral) plane images of the sample in a corresponding plurality of the selected i-th fields of view, FOVi, wherein for each the i-th field of view, FOVi, there is imaging at a plurality of the selected j-th differential imaging or focusing distances, xcex94zij; (g) constructing a fused focused cube (spectral) image from the plurality of the focused cube (spectral) plane images and empirically determined spectral background parameters, Bi, obtained from the plurality of the focused cube (spectral) plane images; (h) acquiring and constructing additional fused focused cube (spectral) images of the sample in other fields of view, FOVi, for a plurality of the differential imaging/focusing distances, xcex94zij, by repeating the step (c) through the step (g), until selected sample viewing/imaging range is imaged; (i) applying at least one image analysis algorithm to data of the plurality of the fused focused cube (spectral) images, for identifying spectral fingerprints relating to physicochemical characterization of the sample; and (j) repeating the step (c) through the step (i) over a period of time spanning a multiple of a pre-determined time interval, xcex94t, whereby following each predetermined time interval, xcex94t, generating a statistical analysis report describing time variation of physicochemical and spectral imaging characteristics of the particulate sample.
Two types of xe2x80x98spectral distancesxe2x80x99 are used in the description of the method for focus-fusion multi-layer spectral imaging of the present invention. The first type of spectral distance is the Blob neighborhood spectral distance parameter, Ds, defined as the physical, geometrical distance encompassing a number of selected neighboring Blobs in the same Blob neighborhood as a particularly identified sharp or focused Blobb, referred to as Blobs, for s=1 to any number, S, of sharp or focused Blobs, each having central gravity position coordinates (xs, ys). In a Blob neighborhood, the sharp or focused Blobs, and the selected neighboring or neighborhood Blobsb are considered suitable for including in the process of constructing focused cube (spectral) plane images and in constructing xe2x80x98fusedxe2x80x99 focused cube (spectral) images of a sample. Blob neighborhood spectral distance parameter, Ds, is determined according to criteria specific to a particular application, for example, in-situ, while the detecting, imaging, and analysis of Blobs is in progress, and is a function of particle spectral information and parameters, such as pixel intensity, signal-to-noise ratio (S/N) of imaging or spectral signals corresponding to Blob and non-Blob pixels, image sharpness, and spectral fingerprints relating to spectral emission patterns of individual particles.
The second type of spectral distance is the inter-Blob spectral distance, xcex94dbs, defined as the physical geometrical distance between an identified Blob, Blobb, having position coordinates (xb, yb), and a sharp or focused Blob, Blobs, having position coordinates (xs, ys), both located in the same i-th field of view, FOVi, at the same j-th differential imaging or focusing distance, xcex94zij.
In the present invention, there is calculating a focus-fusion factor parameter, Fb, from a set of calculated physicochemical and spectral parameters for each identified Blob, Blobb, in the i-th field of view, FOVi, at the j-th differential imaging or focusing distance, xcex94zij, of a cube (spectral) plane image of a sample, by using a formula based on applying fuzzy logic analysis:
Fb=fuzzy logic function [(physicochemical parameters of Blobb), (spectral parameters of Blobb)],
where the physicochemical parameters of Blobb relate to particle chemistry and particle morphology, and the spectral parameters of Blobb relate to spectral fingerprints featured in spectral emission patterns of individual particles of the sample.